Image recognition device and image recognition method

ABSTRACT

An image recognition device includes: a hardware processor that: conducts machine learning, to perform a first process of calculating a plurality of region candidates for a region showing part of an object captured in an image, and a second process of determining a size of each of the region candidates in accordance with the object captured in the image; and determines the region from among the region candidates, using a predetermined criterion.

The entire disclosure of Japanese patent Application No. 2017-115533,filed on Jun. 13, 2017, is incorporated herein by reference in itsentirety.

BACKGROUND Technological Field

The present invention relates to a technology for image recognitionusing machine learning.

Description of the Related Art

Techniques for detecting the position of joints of a person by analyzingan image of the person through machine learning, and estimating theposture of the person from the positions of the joints are disclosed byAdrian Bulat and one other person in “Human pose estimation viaConvolutional Part Heatmap Regression”, [online], p. 2, [search date:May 13, 2017], Internet <URL: https://arxiv.org/pdf/1609.01743>, and ZheCao and three others in “Realtime Multi-Person 2D Pose Estimation usingPart Affinity Fields”, [online], p. 2, [search date: May 13, 2017],Internet <URL: https://arxiv.org/pdf/1611.08050>, for example. Accordingto the techniques disclosed in these literatures, a likelihood map inwhich the likelihoods of joint positions are calculated pixel by pixelis created. Therefore, the amount of calculation is large, and it isdifficult to detect joint positions at high speed.

YOLO (You only look once) is known as one of image recognitiontechniques that require relatively small amounts of calculation. InYOLO, an image is divided into grids (7×7 pixels, for example), and arectangular region (a region showing an object) called a bounding boxcircumscribing the object is set in the grid where the object (such as aperson) exists.

In a case where the recognition target is an object, the range is clear,and accordingly, it is easy to define a bounding box. In a case wherethe recognition target is part of an object, such as the head of aperson, the range is clear, and accordingly, it is easy to define abounding box. However, a joint does not have a clear range, for example.Therefore, it is difficult to define a bounding box, and joints areregarded as points in image recognition.

SUMMARY

An object of the present invention is to provide an image recognitiondevice and an image recognition method capable of reducing the amount ofcalculation and setting a region showing a recognition target in animage even if the range of the recognition target is not clear.

To achieve the abovementioned object, according to an aspect of thepresent invention, an image recognition device reflecting one aspect ofthe present invention comprises: a hardware processor that: conductsmachine learning, to perform a first process of calculating a pluralityof region candidates for a region showing part of an object captured inan image, and a second process of determining a size of each of theregion candidates in accordance with the object captured in the image;and determines the region from among the region candidates, using apredetermined criterion.

BRIEF DESCRIPTION OF THE DRAWINGS

The advantages and features provided by one or more embodiments of theinvention will become more fully understood from the detaileddescription given hereinbelow and the appended drawings which are givenby way of illustration only, and thus are not intended as a definitionof the limits of the present invention:

FIG. 1 is a functional block diagram showing an image recognition deviceaccording to an embodiment;

FIG. 2 is a block diagram showing the hardware configuration of theimage recognition device shown in FIG. 1;

FIG. 3 is a flowchart for explaining a prediction/recognition phase ofmachine learning, which is executed in a case where one person iscaptured in an image;

FIG. 4 is a schematic diagram showing an example of an image in whichone person is captured;

FIG. 5 is a schematic diagram showing an image including five rightshoulder joint region candidates;

FIG. 6 is a schematic diagram showing an image including a rightshoulder joint region;

FIG. 7 is a flowchart for explaining a prediction/recognition phase ofmachine learning, which is executed in a case where two or more personsare captured in an image;

FIG. 8 is a schematic diagram showing an example of an image in whichtwo or more persons are captured;

FIG. 9 is a schematic diagram showing an image including two or morerectangular region candidates;

FIG. 10 is a schematic diagram showing an image including seven rightshoulder joint region candidates;

FIG. 11 is a schematic diagram showing an image including rectangularregions circumscribing persons; and

FIG. 12 is a schematic diagram showing an image including rectangularregions circumscribing persons and right shoulder joint regions.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, one or more embodiments of the present invention will bedescribed in detail with reference to the drawings. However, the scopeof the invention is not limited to the disclosed embodiments. In thedrawings, like components are denoted by like reference numerals. In thedescription below, explanation of components that have already beendescribed will not be repeated. In this specification, general terms areaccompanied by reference numerals without suffixes (“image Im”, forexample), and individual components are denoted by reference numeralswith suffixes (“image Im-1”, for example).

FIG. 1 is a functional block diagram showing an image recognition device1 according to an embodiment. The image recognition device 1 is apersonal computer, a smartphone, a tablet terminal, or the like, andincludes an image inputter 2, a control processing unit 3, an inputter4, and an outputter 5 as functional blocks.

An image Im (an image Im-1 shown in FIG. 4, for example) is input to theimage inputter 2 from the outside of the image recognition device 1. Theimage inputter 2 sends the input image Im to the control processing unit3.

The control processing unit 3 includes a storage unit 31 and a machinelearning unit 32 as functional blocks. The image Im sent from the imageinputter 2 is stored into the storage unit 31.

The machine learning unit 32 detects the position of each joint of aperson 101 (such as a person 101-1 shown in FIG. 4) captured in theimage Im, and, in accordance with the detected position of each joint,estimates the position of the person 101. In the embodiment, the rightshoulder joint is described as an example of a joint. Since a knowntechnique can be used in estimating a posture of the person 101 inaccordance with the detected position of each joint, explanation of theestimating process is not made herein.

Machine learning generally includes a learning phase (creating a modelthrough learning) and a prediction/recognition phase (obtaining a resultby applying the model to data). The machine learning unit 32 thatconducts machine learning includes a processor 321 and a determiner 322as functional blocks. The processor 321 conducts machine learning toperform a first process of calculating region candidates that arecandidates for regions showing part of the object captured in the image(calculating the right shoulder joint region candidates 105-1 through105-5 shown in FIG. 5), and a second process of determining the size ofeach of the region candidates from the object captured in the image Im.This is the prediction/recognition phase (the learning phase will bedescribed later). In the embodiment, the person 101 is taken as anexample, and the right shoulder joint of the person 101 is described asan example of part of the object. The right shoulder joint region (aright shoulder joint region 107-1 shown in FIG. 6, for example) capturedin the image Im is the region indicating the right shoulder joint.

The determiner 322 determines a region (a right shoulder joint region107) from among the region candidates (right shoulder joint regioncandidates 105) determined by the processor 321, using a predeterminedcriterion. For example, the determiner 322 determines the regioncandidate (a right shoulder joint region candidate 105) having thehighest likelihood to be the region (the right shoulder joint region107). The determiner 322 outputs the center of the region (the rightshoulder joint region 107) as a feature point. The output feature pointindicates the position of the right shoulder joint, and is used inestimating the posture of the person 101 by the machine learning unit32.

The inputter 4 is a device that inputs commands (instructions), data,and the like from the outside to the image recognition device 1. As willbe described later, in a case where the machine learning unit 32 is madeto execute the prediction/recognition phase (the first process and thesecond process described above), if one person is captured in the image,the operator of the image recognition device 1 inputs a command forsetting “one person” in the machine learning unit 32 to the inputter 4.If two or more persons are captured in the image, the operator inputs acommand for setting “two or more persons” in the machine learning unit32 to the inputter 4. The outputter 5 is a device that outputs theresult (the posture of the person 101 captured in the image Im, forexample) of image recognition performed by the image recognition device1.

FIG. 2 is a block diagram showing the hardware configuration of theimage recognition device 1 shown in FIG. 1. The image recognition device1 includes a central processing unit (CPU) 1 a, a random access memory(RAM) 1 b, a read only memory (ROM) 1 c, a hard disk drive (HDD) 1 d, aliquid crystal display 1 e, an image input interface 1 f, a keyboard andthe like 1 g, and a bus 1 h that connects these components. The liquidcrystal display 1 e is hardware that forms the outputter 5.

Instead of the liquid crystal display 1 e, an organicelectroluminescence (EL) display, a plasma display, or the like may beused. The image input interface if is hardware that forms the imageinputter 2. The keyboard and the like 1 g is hardware that forms theinputter 4. Instead of the keyboard, a touch panel may be used.

The HDD 1 d is hardware that forms the storage unit 31. The HDD 1 dstores programs for implementing the functional blocks of the processor321 and the determiner 322, respectively. These programs are expressedwith the use of the definitions of the functional blocks. The determiner322 and a determination program are now described as an example. Thedeterminer 322 determines a region from among the region candidates,using the predetermined criterion. The determination program is aprogram for determining a region from among the region candidates withthe use of the predetermined criterion.

These programs are stored beforehand in the HDD 1 d, but the presentinvention is not limited to this. For example, a recording medium (anexternal recording medium such as a magnetic disk or an optical disk) inwhich these programs are recorded is prepared, and the programs recordedin the recording medium may be stored into the HDD 1 d. Alternatively,these programs may be stored in a server connected to the imagerecognition device 1 via a network. In such a case, these programs maybe sent to the HDD 1 d via the network, and be stored into the HDD 1 d.These programs may be stored into the ROM 1 c, instead of the HDD 1 d.The image recognition device 1 may include a flash memory, instead ofthe HDD 1 d, and these programs may be stored into the flash memory.

The CPU 1 a reads these programs from the HDD 1 d, loads the programs inthe RAM 1 b, and executes the loaded programs, to form the processor 321and the determiner 322. However, as for the functions of the processor321 and the functions of the determiner 322, a part or all of eachfunction may be realized by a process to be performed by a digitalsignal processor (DSP), instead of or in combination with a process tobe performed by the CPU 1 a. Likewise, part or all of each function maybe realized by a process to be performed by a dedicated hardwarecircuit, instead of or in combination with a process to be performed bysoftware.

Flowcharts according to these programs (a processing program, adetermination program, and the like) to be executed by the CPU 1 a arethe later described flowcharts shown in FIGS. 3 and 7.

In the learning phase, the machine learning unit 32 learns to detect theright shoulder joint region, using a large number (at least two) ofimages in which one or more persons are captured and a regionsurrounding the right shoulder joint of a person (or a regionoverlapping the right shoulder joint of a person) is set. At this stage,the machine learning unit 32 performs a process of determining the sizeof each side of the right shoulder joint region to be 0.4 times largerthan the rectangular region (a bounding box) circumscribing the head ofa person captured in the images. The machine learning unit 32 performsthis process on the large number (at least two) of images. Byconstructing such a learning model, the machine learning unit 32 cancalculate right shoulder joint region candidates from an image showingone or more persons, and also predict the size of each side of each ofthe right shoulder joint region candidates (the predicted size is 0.4times the bounding box surrounding the head) from the image, withoutdetecting the head of a person captured in the image (that is, thebounding box around the head is not set), in the prediction/recognitionphase.

The learning phase to be executed by the machine learning unit 32 isgeneralized as follows. The machine learning unit 32 conducts machinelearning to perform a third process of detecting a region showing partof an object by using images in which the region is set and the objectis captured, and a fourth process that is a process of determining thesize of the region from the object captured in the image (for example, aprocess of determining the size of the region to be a size having apositive correlation with the size that can be defined by the objectcaptured in the image is performed on each of the images). In thismanner, the machine learning unit 32 constructs the learning model inadvance.

Next, the prediction/recognition phase is described. The imagerecognition device 1 performs different processes in a case where oneperson 101 is captured in the image Im, and in a case where two or morepersons 101 are captured in the image Im. Referring now to FIG. 3, theformer case is described. FIG. 3 is a flowchart for explaining theprediction/recognition phase of machine learning, which is executed in acase where one person 101 is captured in the image Im.

FIG. 4 is a schematic diagram showing an example of the image Im-1 inwhich one person 101 is captured. The number of persons 101 in the imageIm-1 is one (singular). The image Im-1 is stored into the storage unit31 via the image inputter 2. The image Im-1 shown in FIG. 4 is an imageIm that is to be subjected to the prediction/recognition phase ofmachine learning.

The operator of the image recognition device 1 operates the inputter 4,to input a command for setting information indicating that one person101 is captured in the image Im-1, in the processor 321. As a result,information indicating that one person 101 is captured in the image Im-1is set in the processor 321 (step S1 in FIG. 3). The machine learningunit 32 recognizes that one person 101 is captured in the image Im-1,and executes the prediction/recognition phase.

The processor 321 reads the image Im-1 stored in the storage unit 31,inputs the image Im-1 to the above described learning model, calculatesright shoulder joint region candidates 105 (step S2 in FIG. 3), andcalculates the positional information, the size, and the likelihood(probability) for each of the right shoulder joint region candidates 105(step S3 in FIG. 3). The size is the size of one side of eachcorresponding right shoulder joint region candidate 105. In theprocessing in steps S2 and S3, a convolutional neural network (CNN) isused, for example. In this example, the number of right shoulder jointregion candidates 5 is five. FIG. 5 is a schematic diagram showing theimage Im-1 including the five right shoulder joint region candidates105-1 through 105-5. The right shoulder joint region candidates 105 arecandidates for the right shoulder joint region 107. Positionalinformation, a size, and a likelihood (probability) are given to each ofthe right shoulder joint region candidates 105. The positionalinformation indicates the position of the corresponding right shoulderjoint region candidate 105 in the image Im-1. If the likelihood is high,the possibility that the right shoulder joint region candidate 105 isthe right shoulder joint region 107 is high. If the likelihood is low,the possibility that the right shoulder joint region candidate 105 isthe right shoulder joint region 107 Is low. In FIG. 5, the densities ofhatching in the right shoulder joint region candidates 105 indicate thedegrees of likelihood. If the density of hatching is high, thelikelihood is high. If the density of hatching is low, the likelihood islow.

The determiner 322 determines that the right shoulder joint region 107is the right shoulder joint region candidate 105 having the highestlikelihood among the right shoulder joint region candidates 105 (thefive right shoulder joint region candidates 105-1 through 105-5 shown inFIG. 5) calculated in step S3 (step S4 in FIG. 3). FIG. 6 is a schematicdiagram showing the image Im-1 including the right shoulder joint region107. The right shoulder joint region candidate 105-2 is determined to bethe right shoulder joint region 107-1 of the person 101-1.

As described above, the image recognition device 1 predicts anddetermines the size of the right shoulder joint region 107 showing theright shoulder joint of the person 101-1 captured in the image Im, fromthe external appearance of the person 101-1 captured in the image Im.Thus, the right shoulder joint region 107 showing the right shoulderjoint can be set in the image Im, even if the range of the rightshoulder joint is not clear. Further, in the image recognition device 1,the right shoulder joint is not detected pixel by pixel, but is detectedfor each right shoulder joint region candidate 105 shown in FIG. 5.Thus, the amount of calculation in machine learning can be reduced.

Referring now to FIG. 7, the process to be performed in a case where twoor more persons 101 are captured in the image Im is described. FIG. 7 isa flowchart for explaining the prediction/recognition phase of machinelearning, which is executed in a case where two or more persons 101 arecaptured in the image Im.

FIG. 8 is a schematic diagram showing an example of an image Im-2 inwhich two or more persons 101 are captured. In this example, the numberof persons 101 captured in the image Im-2 is two. The image Im-2 isstored into the storage unit 31 via the image inputter 2. The image Im-2shown in FIG. 8 is an image Im to be subjected to theprediction/recognition phase of machine learning.

The operator of the image recognition device 1 operates the inputter 4,to input a command for setting information indicating that two or morepersons 101 are captured in the image Im-2, in the processor 321. As aresult, information indicating that two or more persons 101 are capturedin the image Im-2 is set in the processor 321 (step S11 in FIG. 7). Themachine learning unit 32 recognizes that two or more persons 101 arecaptured in the image Im-2, and executes the prediction/recognitionphase.

The processor 321 conducts machine learning, to perform a process ofcalculating rectangular region candidates 109 for each of the persons101-1 and 101-2 that are captured in the image Im-2 (step S12 in FIG.7). FIG. 9 is a schematic diagram showing the image Im-2 including twoor more rectangular region candidates 109. In this example case, thenumber of rectangular region candidates 109 is five. The rectangularregion candidates 109-1, 109-2, and 109-3 are candidates for arectangular region 111 (a bounding box) circumscribing the person 101-1.The rectangular region candidates 109-4 and 109-5 are candidates for therectangular region 111 (a bounding box) circumscribing the person 101-2.

The processor 321 inputs the image Im-2 to the above described learningmodel, calculates the right shoulder joint region candidates 105 of thepersons 101 included in the image Im-2 (step S13 in FIG. 7), andcalculates the positional information, the size, and the likelihood(probability) of each of the right shoulder joint region candidates 105(step S14 in FIG. 7). The size is the size of one side of eachcorresponding right shoulder joint region candidate 105. In theprocessing in steps S13 and S14, a convolutional neural network (CNN) isused, for example. In this example, the number of right shoulder jointregion candidates 5 is seven. FIG. 10 is a schematic diagram showing theimage Im-2 including the seven right shoulder joint region candidates105-1 through 105-7. As described above in the case where there is onlyone person 101 captured in the image Im, positional information and alikelihood (probability) are given to each of the right shoulder jointregion candidates 105. In FIG. 10, the densities of hatching in theright shoulder joint region candidates 105 indicate the degrees oflikelihood, as in FIG. 5. If the density of hatching is high, thelikelihood is high. If the density of hatching is low, the likelihood islow.

Through machine learning, the processor 321 calculates a classificationprobability indicating to which one of the five rectangular regioncandidates 109-1 through 109-5 shown in FIG. 9 a right shoulder jointregion candidate 105 belongs (step S15 in FIG. 7). The processor 321performs this calculation for each of the seven right shoulder jointregion candidates 105-1 through 105-7 shown in FIG. 10. Where the rightshoulder joint region candidate 105-1 is taken as an example, theprocessor 321 calculates the classification probability indicating whichone of the five rectangular region candidates 109-1 through 109-5 theright shoulder joint region candidate 105-1 belongs.

The processor 321 determines the rectangular region 111-1 circumscribingthe person 101-1 from among the five rectangular region candidates 109-1through 109-5 shown in FIG. 9, and determines the rectangular region111-2 circumscribing the person 101-2 from among the five rectangularregion candidates 109-1 through 109-5 (step S16 in FIG. 7). FIG. 11 is aschematic diagram showing the image Im-2 including the rectangularregions 111 circumscribing the person 101. The rectangular regioncandidate 109-2 is determined to be the rectangular region 111-1circumscribing the person 101-1, and the rectangular region candidate109-5 is determined to be the rectangular region 111-2 circumscribingthe person 101-2. In this determination, a non-maximum suppressionprocess is used, for example. Specifically, the processor 321 selects arectangular region candidate 109 having a high likelihood from among thefive rectangular region candidates 109-1 through 109-5 for therectangular region circumscribing the person 101-1. If the selectedrectangular region candidate 109 does not overlap any other rectangularregion candidate 109 having a higher likelihood than that, the processor321 determines the selected rectangular region candidate 109 to be therectangular region 111-1 circumscribing the person 101-1. The processor321 determines the rectangular region 111-2 circumscribing the person101-2 in the same manner as above.

The determiner 322 determines a right shoulder joint region 107 for eachof the persons 101-1 and 101-2 (step S17 in FIG. 7). FIG. 12 is aschematic diagram showing the image Im-2 including the rectangularregions 111 circumscribing the persons 101 and the right shoulder jointregions 107. The right shoulder joint region candidate 105-2 isdetermined to be the right shoulder joint region 107-1 of the person101-1, and the right shoulder joint region candidate 105-7 is determinedto be the right shoulder joint region 107-2 of the person 101-2.

A method of determining the right shoulder joint regions 107 is nowdescribed in detail. For each of the seven right shoulder joint regioncandidates 105-1 through 105-7 shown in FIG. 10, the determiner 322calculates the product of the classification probability that the rightshoulder joint region candidate 105 belongs to the rectangular region111-1 (the rectangular region candidate 109-2) and the likelihood of theright shoulder joint region candidate 105. The determiner 322 thendetermines that the right shoulder joint region candidate 105 with thelargest product is the right shoulder joint region 107-1 belonging tothe rectangular region 111-1 (or the right shoulder joint region 107-1of the person 101-1). Likewise, for each of the seven right shoulderjoint region candidates 105-1 through 105-7, the determiner 322calculates the product of the classification probability that the rightshoulder joint region candidate 105 belongs to the rectangular region111-2 (the rectangular region candidate 109-5) and the likelihood of theright shoulder joint region candidate 105. The determiner 322 thendetermines that the right shoulder joint region candidate 105 with thelargest product is the right shoulder joint region 107-2 belonging tothe rectangular region 111-2 (or the right shoulder joint region 107-2of the person 101-2).

The classification probability that a right shoulder joint regioncandidate 105 belongs to the rectangular region 111-1 is theclassification probability that the right shoulder joint regioncandidate 105 belongs to the rectangular region candidate 109-2. Theclassification probability that a right shoulder joint region candidate105 belongs to the rectangular region 111-2 is the classificationprobability that the right shoulder joint region candidate 105 belongsto the rectangular region candidate 109-5. These classificationprobabilities have already been calculated in step S15.

In a case where the number of persons 101 captured in the image Im-2 istwo or larger, the right shoulder joint regions 107 of the two or morepersons cannot be determined only from the likelihoods. Therefore, inthe image recognition device 1, a right shoulder joint region 107 isdetermined for each of the two or more persons 101 captured in the imageIm-2, in accordance with the classification probabilities and thelikelihoods (the right shoulder joint region 107 belonging to arectangular region 111 is determined for each of the two or morerectangular regions 111).

Although embodiments of the present invention have been described andillustrated in detail, the disclosed embodiments are made for purposesof illustration and example only and not limitation. The scope of thepresent invention should be interpreted by terms of the appended claims.

What is claimed is:
 1. An image recognition device comprising: ahardware processor that: conducts machine learning, to perform a firstprocess of calculating a plurality of region candidates for a regionshowing a part of an object captured in an image, the part of the objectbeing a portion where an outline of a recognition target is not clear,the part of the object that is the recognition target being recognizedas a bounding box, and a second process of determining a size of each ofthe region candidates in accordance with the object captured in theimage; and determines the region from among the region candidates, usinga predetermined criterion.
 2. The image recognition device according toclaim 1, wherein the hardware processor stores a learning model inadvance, and performs the first process and the second process by usingthe learning model, the learning model being constructed by the hardwareprocessor performing a third process of detecting the region by using aplurality of images in which the region is set and the object iscaptured, and a fourth process of performing, for each of the images, aprocess of determining a size of the region in accordance with theobject captured in the image, the third process and the fourth processbeing performed through machine learning.
 3. An image recognition devicecomprising: a hardware processor that: conducts machine learning, toperform a first process of calculating a plurality of region candidatesfor a region showing part of an object captured in an image, and asecond process of determining a size of each of the region candidates inaccordance with the object captured in the image, and determines theregion from among the region candidates, using a predeterminedcriterion; and an inputter that receives an input of a command forsetting information indicating that the object is a single object in thehardware processor from an operator of the image recognition device,when the object captured in the image is a single object in a case wherethe hardware processor is made to perform the first process and thesecond process, wherein the hardware processor further: conducts machinelearning, to perform a process of calculating a likelihood that a regioncandidate is the region, the process being performed for each of theregion candidates, and determines that the region candidate having thehighest likelihood among the region candidates is the region.
 4. Theimage recognition device according to claim 1, further comprising aninputter that receives an input of a command for setting informationindicating that the object is at least two objects in the hardwareprocessor from an operator of the image recognition device, when theobject captured in the image is at least two objects in a case where thehardware processor is made to perform the first process and the secondprocess, wherein the hardware processor conducts machine learning, toperform a process of calculating a plurality of rectangular regioncandidates for a rectangular region circumscribing the object, for eachof the at least two objects captured in the image, conducts machinelearning, to calculate a classification probability indicating which ofthe rectangular region candidates a region candidate belongs to, foreach of the region candidates, determines the rectangular regioncircumscribing the object from among the rectangular region candidates,for each of the at least two objects, conducts machine learning, toperform a process of calculating a likelihood that a region candidate isthe region, for each of the region candidates, and determines the regionbelonging to the rectangular region from among the region candidates,for each of the two or more rectangular regions, in accordance with theclassification probabilities of the region candidates belonging to therectangular region candidate determined to be the rectangular region,and the likelihoods of the respective region candidates.
 5. An imagerecognition method comprising: conducting machine learning, to perform afirst process of calculating a plurality of region candidates for aregion showing a part of an object captured in an image, the part of theobject being a portion where an outline of a recognition target is notclear, the part of the object that is the recognition target beingrecognized as a bounding box, and a second process of determining a sizeof each of the region candidates in accordance with the object capturedin the image; and determining the region from among the regioncandidates, using a predetermined criterion.
 6. A non-transitorycomputer readable medium storing instructions to cause aprocessor-controlled apparatus to perform the method of claim 5.